A message from psychologists to economists : mere predictability doesn ’ t matter like it should (

Stephen J. Gould (Pittsburgh, 3/3/97) recently defined humans as ‘the primates who tell stories.’ This paper reviews evidence for a more radical definition as ‘the primates whose cognitive capacity shuts down in the absence of a story’ when attempting to incorporate probabilistic information to make a coherent probabilistic inference. Thus, people cannot conform (‘descriptively’) to the standard expected utility (EU) model of economic decision making, given that probabilities often cannot be combined either implicitly or explicitly in the absence of a good, clearly relevant story justifying the combination. Moreover, that inability severely limits the standard EU model for use in prescriptive decision making. ©1999 Elsevier Science B.V. All rights reserved. People have a great deal of difficulty appreciating statistical contingency in the absence of a causal story that makes the contingency ‘reasonable.’ In particular, people often do not understand on a purely statistical basis the relationship between properties that describe sets versus those that describe members of these sets, particularly when thinking about human behavior. For example, people often ‘underutilize’ base rates—that is, the simple extent of the sets or classes about which inference are to be made—in the absence of a causal story; thus, in deciding whether a particular blue taxi was responsible for a particular hit-and-run accident in visibility poor enough to preclude definite identification of the taxi color, people will often ignore the base rate information that 90 percent of the accidents are due to blue taxis because 90 percent of taxis in town are blue; when, however, people are told that 90 percent of accidents are due to blue taxis because the blue taxi company hires inferior drivers and does not train them well, this base rate information is incorporated into judgments about the particular accident (given it creates a story about why we should q This paper was first presented at a conference of Psychology and Economics, Faculty of Commerce and Business Administration, University of British Columbia, Vancouver, BC, Canada, 6/6/97–6/7/97. ∗ Fax: +1-412-268-6938. 0167-2681/99/$ – see front matter ©1999 Elsevier Science B.V. All rights reserved. PII: S0167 268 1 (99 )00024 -4 30 R.M. Dawes / J. of Economic Behavior & Org. 39 (1999) 29–40 suspect a particular blue taxi; see, e.g. Ajzen, 1977). Conversely, even some professorial researchers in psychology fail to understand that base rate inference is based on a positive correlation between overall frequency and individual outcomes, and that because correlation is symmetric, it is possible to infer base rates from single observations; that inference has been termed a ‘false consensus effect’ if the observation happens to be about one’s own behaviors or predilections or attitudes. In contrast, people will accept a causal story of a single incident or event as convincing, even if there is absolutely no general contingency established between the elements that are claimed to be ‘causal’ in the story and those that are claimed to follow from these causes. I chose to oversimplify in the statements I have just made, because some people some of the time do accept some contingency without causality. For example, on hearing that in general one type of medical procedure for a particular problem is twice as successful as another type, we generally opt for the former type—even in the absence of any understanding whatsoever of the details of the procedures and the conditions. It works. I want it. In such medical contexts, we accept as quite natural the inference from the group of people (taking a vaccine, having the operation, ingesting the drug) to us as a single individual—or to our children. In contrast, however, most of us do not accept the statement that since most people in our income category vote Republican (or Democrat) we also are likely to vote Republican (or Democrat) in the future. Our political beliefs are clearly influenced by factors that have nothing to do with our incomes, and many of us would find it almost insulting to be told that whether we know it or not we are ‘influenced’ by our financial status. Most of us regard our political beliefs as ‘basic attitudinal baggage,’ (Abelson, 1986, 1988) to which we are committed for deep philosophical and personal reasons. Even if we were told that the inference from income to individual party identification was stronger than the inference concerning the success of the medical procedure, we would find this former inference (many of us would anyway) strange and unacceptable. Pure financial concerns cannot ‘cause’ our beliefs. On the other hand, there must be something about basic human physiology that led one treatment to be twice as successful as the other, and that would therefore ‘cause’ it to be more likely to be successful for ourselves. The point, however, is that whether we are consistent or not, our failure to rely on a probabilistic contingency when it is not embedded in a causal story violates standard neoclassical economic theory. That theory is based on the valence (either inferred or directly estimated) of outcomes and their probabilities (objective or subjective). It does not matter how the probabilities come about. An estimated probability is an estimated probability, which will affect the estimate of the expected utility of an outcome. The same estimated probability should not affect our behavior dependent on whether or not it is wrapped up in a causal story. But it does. For example, base rate neglect is well known. In fact, it is so well known that now a common way to attempt to make a mark in the field of behavioral decision making is to state that it is overblown (Koehler, 1996), to find some context in which it does not occur, or even to question the applicability of standard probability theory: for example, by claiming that probabilities cannot be applied to single events as opposed to collections of events (Gigerenzer, 1996), or that only relative frequencies make sense psychologically (Cosmides and Tooby, 1994), or even that it is by definition impossible to make an ‘epistemic’ error in R.M. Dawes / J. of Economic Behavior & Org. 39 (1999) 29–40 31 assessing probabilities—because like Popeye we are what we are, although ‘ontologically’ the laws of probability may be valid even if no one existed (Cohen, 1981). That base rate neglect is alive and well can be observed in the current debate in clinical psychology over ‘recovered repressed memory,’ often leading to the discovery of remarkable and severe abuse in satanic cult situations. The argument is that relatively high base rate distressing conditions—such as a negative physical selfimage, or an eating disorder—can be diagnostic of an extremely low base rate ‘causal mechanisms’ (such as being raised in a satanic cult, which may have base rate probability zero). That is just flat out irrational. 1 Diagnosing this causal condition on the basis of the problem is equivalent to claiming it is a necessary condition for the problem to occur. But it cannot logically be a necessary condition when its base rate is so much lower. Nevertheless some clinicians write that it is even unethical not to ‘search for possible etiology involving satanic cult ritual abuse or at least incestuous sexual abuse’ when finding problems such as a negative self-image or eating disorders. (In fact, there is not even any evidence that the inference works in the other direction either, but that is another matter.) Note, however, how compelling the ‘story’ is. Suppose you have been abused sexually day after day after day when you are an infant or child and then hypnotized to forget it, and perhaps even programmed by the satanic cult people to turn to their beck and call if you receive certain ‘triggering messages’ (such as a postcard saying we love you and look forward to seeing you—actual example); then of course you would not feel very good about your body, and you may seek solace in excess eating (or drinking or ingestion of legal or illegal drugs). The base rate experimental findings are quite consistent with the causal story idea. The fact that 90 percent of the taxis in an area are blue does not appear to affect the judgment about whether a particular taxi that might be green but could possibly be blue is in fact blue or green. On the other hand, the story that 90 percent of the accidents are caused by blue taxis because the blue taxi company does not screen its drivers well or train them adequately does affect the judgment. Causality (Ajzen, 1977; Bar-Hillel, 1980) makes the base rate relevant, and may even lead to absolutely bizarre judgments. For example, if people dream in color with probability 0.80, and if there is no relationship between the types of dreams of pairs of sexual partners, then in 64 percent of the pairs both will dream in color, in 4 percent neither will dream in color, and in 32 percent one will dream in color and one will not. That follows from elementary probability theory; 0.8 × 0.8 = 0.64; 0.2 × 0.2 = 0.04, and 0.2 × 0.8 + 0.8 × 0.2 = 0.32. After being told, however, that 68 percent of pairs are concordant and that one dreams in color, judges believe that the probability that the other dreams in color as well is 0.68, not 0.80. Somehow, it is easy to construct a story that ‘like seeks like’—and that the type of dreams people have reflects some type of deep psychological characteristics that others somehow appreciate when seeking partners. At least, it is easy to make up such a story in a society that has yet to wean itself from the fantasy life of Sigmund Freud. Note that what is happening in the examples is that the probabilistic information easily available to the judges is not incorporated into a final probabilistic judgment. When such incorporation fails, then the final probability judgment is adversely affected. Each component prior to combining information is, however, not affected. In contrast, a story, which 1 Since P(A/B) = P(A and B)/P(B) and P(A and B) ≤ P(A), then P(A/B) ≤ P(A)/P(B). 32 R.M. Dawes / J. of Economic Behavior & Org. 39 (1999) 29–40 provides causal information, creates a cohesion among the elements of the inference that can otherwise be lacking, with the result that some of these elements are underutilized or even ignored completely. In fact, a cause or a good narrative often provides causal information that has an effect on all potential outcomes, rather than specifying a subset to which a probability applies. For example, we can imagine that in general ‘like seeks like’ in dreamers, and that selection and training may affect driving. Thus, as Bar-Hillel (1990) has demonstrated, people attend to base rate considerations that can possibly be construed (again, through a ‘good story’) to have some effect on all objects of judgment (in her terms, are incorporated into peoples’ priors). In contrast, base rates that simply make it more or less likely to observe a particular instance, one whose characteristics are defined totally independent of the base rates, tend to be ignored. Yet, each type of base rate is equally important in reaching a rational judgment. Bayes’s theorem does not distinguish between base rates that change probabilities by affecting the ‘mix’ of particular instances versus those that appear to ‘nudge’ all in one direction or another. A conditional probability is a conditional probability. One way of explaining why there is a relationship between base rates and single occurrences is by pointing out that there is a correlation between them. This correlation can be computed by regarding one variable as a vector of zeros and ones and another variable as a vector of base rates. More ones will be associated with higher base rates, more zeros with lower base rates. In fact, it is very easy to figure out what the correlation should be. (See Dawes, 1989.) Moreover, it turns out that a standard regression analysis yields a result that the predicted value of the zero, one variable is precisely the base rate value with which it is paired. But correlation is symmetric. Not so, according to some of our own colleagues—who argue that if there is a relationship between a subject’s own response and her estimate of base rate responding, the subject has evidenced an ‘egoistic’ bias termed the ‘false consensus effect’ (Marks and Miller, 1987). The ‘argument’ made for the falsity and egoism of this effect is to point out that there is a positive relationship between the error—defined as the difference between the population value and the estimate—and the subject’s own response. That is not a good argument, because there is always a positive correlation between an unbiased estimate of the population value and the discrepancy between that estimate and the true population value. How can that be when the estimate is ‘unbiased?’ The answer is that it is unbiased because its expectation is the population value, but the discrepancy between a particular value sampled and the actual population value must in general be in the direction of the value sampled. For example, consider that two of us attempt to assess the average weight of college students in a particular institution by randomly sampling five students. The actual average of our sample weights is an unbiased estimate of the mean population weight. But the discrepancy between this average and any constant value will be in the direction of the average, and in particular the discrepancy between the average and the actual population value will be in the direction of the average. Let me express this conclusion in another manner. In the original study proclaiming the ‘false consensus effect,’ Ross et al. (1977) asked Stanford students to walk around the Stanford campus with a big sign board reading ‘Repent!’ Some agreed to do so and some declined. Subsequently, all of the subjects were asked to estimate the proportion of Stanford students who would agree. Those who agreed estimated on the average that R.M. Dawes / J. of Economic Behavior & Org. 39 (1999) 29–40 33 about 62 percent would, while those who declined estimated that on the average 29 percent would. Well, clearly not everyone can be correct, and the direction of the error from the true proportion (even though we do not happen to know its value) is in the direction of the subject’s own behavior. Hence, the ‘false consensus.’ But consider this problem from a Bayesian perspective. Two people are asked to estimate the proportion of blue versus red chips in a book bag, and one person draws a blue chip while the other person draws a red chip. The best judgment of these people should not be equivalent. Consider, for example, that the judgments were the same; then the sampling of the single chip would not affect the estimate, which would mean that the prior belief following the first draw should be the same as that following no draws, which should then imply that the belief should not be changed by the second draw, and so on. Eventually a sample of 100,000 should have no effect on belief. (Happily, probability theory proclaims the same results whether we think of large samples in terms of the entire sample or in terms of successive samples of size 1; e.g. readers may wish to convince themselves that the probability of drawing 5 spades at random from a deck of cards is [13/52] × [12/51] × [11/50] × [10/49] × [9/48], which is exactly equal to the number of ways 13 things can be selected 5 at a time divided by the number of ways 52 things can be selected 5 at a time.) As the believer in this false consensus effect may point out, my reasoning has to do with random drawing—but the subject herself or himself is not ‘random.’ True, but why is the subject for himself or herself any more or less diagnostic of what Stanford students would do than the previous subject would be? And certainly we would expect estimates to change on the basis of being told what someone else did, or we are once again in the problem of being unaffected by samples of any size whatsoever. I started talking about this problem in 1989. What I want to convey in this paper is that many colleagues did not believe me until I provided examples involving small sample sizes. At that point, these colleagues conceded that I was correct but only for situations in which the subject constituted a ‘significant’ portion of the small group—because then the subject’s own response would have a causal influence on the base rate in the group. When, I was told, I started dealing with large groups, my examples would show that there should be no inference at all based on a single sample of size 1. Well, of course not. In the first place the correlation between base rates and single outcomes is unaffected by sample size; also, denying that a sample of size 1 has any effect except in a finite sampling context would lead to the denial that a sample of any size would have an effect. The point is, however, that as I first talked about my own response as diagnostic, I used small sample statistics to demonstrate that the subject was better off attending to their own response than not, and the sophisticated psychologists to whom I talked immediately interpreted my results in causal terms and denied that they were in fact true when no causal explanation could be devised. For example, consider Table 1, taken from my 1989 article, which present hypothetical 1,0 (e.g. ‘yes’/‘no’) responses of three people to two items (e.g. ‘I feel that my ideas may turn into insects,’ and ‘I look out for my rights.’) Colleagues accepted my conclusion that to minimize mean square error (MSE) people should assign an estimate of 5/9 to responses they themselves endorsed and 4/9 to those they did not (leading to an MSE of 2/81), rather than assign the base rate 1/2 to each (leading to an MSE of 1/36). But then these colleagues claimed that this result held because ‘there were only three subjects and two items’ involved. (See App. A for a derivation of the 4/9, 5/9 estimates using both a Bayesian approach and 34 R.M. Dawes / J. of Economic Behavior & Org. 39 (1999) 29–40